Block-Coordinate Frank-Wolfe Optimization for Structural SVMs
نویسندگان
چکیده
We propose a randomized block-coordinate variant of the classic Frank-Wolfe algorithm for convex optimization with block-separable constraints. Despite its lower iteration cost, we show that it achieves a similar convergence rate in duality gap as the full FrankWolfe algorithm. We also show that, when applied to the dual structural support vector machine (SVM) objective, this yields an online algorithm that has the same low iteration complexity as primal stochastic subgradient methods. However, unlike stochastic subgradient methods, the block-coordinate FrankWolfe algorithm allows us to compute the optimal step-size and yields a computable duality gap guarantee. Our experiments indicate that this simple algorithm outperforms competing structural SVM solvers.
منابع مشابه
Block-Coordinate Frank-Wolfe for Structural SVMs
We propose a randomized block-coordinate variant of the classic Frank-Wolfe algorithm for convex optimization with block-separable constraints. Despite its lower iteration cost, we show that it achieves the same convergence rate as the full Frank-Wolfe algorithm. We also show that, when applied to the dual structural support vector machine (SVM) objective, this algorithm has the same low iterat...
متن کاملMinding the Gaps for Block Frank-Wolfe Optimization of Structured SVMs
In this paper, we propose several improvements on the block-coordinate Frank-Wolfe (BCFW) algorithm from Lacoste-Julien et al. (2013) recently used to optimize the structured support vector machine (SSVM) objective in the context of structured prediction, though it has wider applications. The key intuition behind our improvements is that the estimates of block gaps maintained by BCFW reveal the...
متن کاملStochastic block coordinate Frank-Wolfe algorithm for large-scale biological network alignment
With increasingly "big" data available in biomedical research, deriving accurate and reproducible biology knowledge from such big data imposes enormous computational challenges. In this paper, motivated by recently developed stochastic block coordinate algorithms, we propose a highly scalable randomized block coordinate Frank-Wolfe algorithm for convex optimization with general compact convex c...
متن کاملPartial Linearization Based Optimization for Multi-class SVM
We propose a novel partial linearization based approach for optimizing the multi-class svm learning problem. Our method is an intuitive generalization of the Frank-Wolfe and the exponentiated gradient algorithms. In particular, it allows us to combine several of their desirable qualities into one approach: (i) the use of an expectation oracle (which provides the marginals over each output class...
متن کاملHADES: Hierarchical Approximate Decoding for Structured Prediction
Many fundamental tasks in machine learning require predicting complex objects rather than a simple yes-no answer or a number. Structured Output Prediction deals with learning such complex objects, which model an inherent structure between interdependent variables. Recently, large margin methods like Structured SVMs (SSVM) have gained popularity to solve this task due to efficient and generalize...
متن کامل